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LECTURE I 

MULTIPLICATION IN BIl^RY 

U - lOllQO 
V - llllil 

101160 

10110b 



u X V - 101011010100 



a - Wi' add and shift multi^>l(E:atlon algorittam 

T (U,V) •■ time (number of ba^ic operations on 
* digits) to nwltiply U and V by 

method or* 

T (n) - max (T (U,V)| i(U) » £(V) - n } 
cr or 
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T^(n) - 0(n ) 
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'f - „. 



nWP:-^5^, 



p - better recursive algorithm using only three 
half length multlplicatlotis 
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Tp(n) - 3Tp(n/2) + 0(n) 
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Turing Machine 
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TWO-WAY READ-WRITE HEAD 
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FLOWCHARTS FOR TM'S 



(j^ 



MOVE 
RIGHT 



MOVE 
UIFT 




cr € L - C tape 

symbols) 



-•^e; 



^^^-^r *p -^^yi«)«^j|^^ ^ - -^-yw-^ 



'--v;..-'.?-s^T^Tp;Fii,»-=- ■ 
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12 m a T.M. , 

X an input word. 
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T (x) = number of instructions executed by TO 
® on X if TO halts; » if TO doesn't halt 
on X. 



S (x) = number of tape squares visited by 
^ head of TO with input x if TO halts; 
00 if TO does not halt. 



(¥)n^(x) = output of TO on X, if any; 
00 if no output. 

T = t^itne S = space co = function 

Church ' s Thesis : 

The effectively (mechanically) computable 
functions and the Turing machine computable 
functions are the same. 

Extended Church's Thesis: 

If a function is computable in time T on 
any reasonable computer model, then it is 
computable in time s polynomial (T) on a 
Turing machine. 
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Inflnitely - often Speed-up Theorem : 

(M. BLUM). Let t: N -+ N be any computable 
function. Then there is a computable function 
C :N -» (0,1} such that 



given any T) computing C one can 
construct an !IJ1' also computing C 
with the property that 

T^Cx) > t(x) and T^, (x) < constant 

for infinitely many x G N. 
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numbers of 
steps 




T^. (^) 



" T_,(x) 



input x 
V is faster than Ul infinitely often. 



W is faster than W infinitely often, 
etc. 



16 Let SD» .aJL,...,!IR. be an ordetly list of 

of All Turttig atehlnes (say in order of the 
sise of their flowcharts). 

Let cp abbreviate 

i SDI. 

Universal HaiaiiM» Ibtmtfm : 

(p. (x), regarded as a function of both 
i and X, is conputable. 



16a PADDIIC LEMtA ; 

Given any program, one dati t>ad it with 
instructions which it never uSes. Thus, 
we obtain 

a new program with the same behavior 
as the old one. 

Hore formally. 
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18a (1) C. U ««i|>fit«Me Cti^liOtt i» the 

Universal Machine Tin.) 



(2) Ifffl , -C., then r ,(e') > t<«fj 

■ Te • -'■■ . t e 

and C (e*) - (by def. of C^). 



■^o .} yt-ii-' 



(3) Say<p » C . then for any e'6PAD(e), 

, 4 '^'J.'':MiiO■': 



and C^(«') - 0. 

So speed-tt^ St by alim^t t*stii« l£ tiie 
input is in PAD(e), and if sb ianadiately 
print output 0. 
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20 Pef. A coi^Mitable t: N •♦ N liB tlme-hoiwt iff 

t € Tline(t^) «iid t(x) ^ i(x). 



Coy. (CooprMviOB Theorem, UrnvtrnKol^Stmifm) 

4 
For any tiaB-liOBMe% C^ 6 Tlme(t ) - Tlne(t). 



iUMftrk: Lots of tiBt-homat fens. 
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closed under 4-, •, exp, conpdsition. 
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Honesty Theorem (HcCr eight, Meyer) 



For every cooputable t, there Is a tine- 
honest t' such that 

Time(t) - TtiBe(t') 
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Staanary: 

For arbitrarily large t^ t* it sii|H|i$fpeii 
that „ .3. 

/ V \ 



GAP 



HONESTY 



COMPRESSION 



i'J :tm^ 
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26 Def. Let f be a conputable {uiotitlon. A 

sequence t-.t.*-** of functlQns is a (spaces^ 

complexl^tY s^H^iye for £ iff 

(1) If qs " f, then S 2: t almost every 
where for some i, 
and (2) For every i, there is a cp = f 
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sxich that 



t ^ S almost everywhere. 



Def . A 6vi\}iKm^ 61 f!^mtimiM- 

(for space) If £ 
(1> p-^£ «|^ .pi fdr all 1» 

(2) for eaeh 1 'th*r# it i j ilieh tha€| 
and (3) p. (x) is a computable function of i and x. 



28 Theorem (Meyer, Schnorr) Every computable 
function has an r.e. complexity sequence. 

Every r.e. complexity sequence is a 
complexity sequence for some 0-1 valued 
computable function. 
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Example: 



Let t. (x) 
1 



2^' 



So t , = log„t almost everywhere. 
i+1 I 1 

Cor. Almost everywhere Speed-up 

(Blum) There is a 0-1 valued computable 
function, c, such that for any T.M. computing 
c there is another T.M. computing c which uses 
exponentially less space at almost all 
arguments. 
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LECTURE II 

S - finite set called the altfhit>et or 
vocabulary J 

an elei^nt ff € E is called a letter . 



E « set of all, fi^te s^uence of 

letters J 

'a 
an eleaent x € S is called H vord . 



Binary operation concatenation , written " • " 

on L : 

jt.y • s^ - ypxd X followed by word y. 

Exasple ; OOl'Ol » 00101 

je(x) « let«th (nuiaber Of odtutreneeas of letter) 
of the word x. 

i(x.y) - jfeQ*) +4<y)v 



X £ T, acts as an identity element under 
concatenation. 



X..X = x«\ = X for all x € S , 
JJ(X) = 0. 

Remark 1 : <i; , • > is the free monoid 
generated by L with identity X. 

Remark 2 : Remark 1 is irrelevant. 

Remark 3 : A. is introduced as a technical 
convenience and could be eliminated in what 
follows at the expense of some minor 
awkwardness. 



A set L C E is called a language. Extending 
concatenation to languages in the usual 

way: 

def. 
L«M, also written LM = 

{x-y I X e L and y € M] 

Example : [0] .[0,1] = {00,01} 

[0,00]. [1,01} = [01,001,0001] 

[0,1,\] '{O.l.M'CO'l'M = ^11 binary words of 

length £ 3 (including \) . 



T'-f^-j 



For X € E , n 6 N, 



x»*»*»x 



x''.^ 



BxmmU : (01) - dlblOl 



SlBiUrly for A c £ 



t , Ul,' ; « - 



A - A*A*<!A 




A - (M 


....:..'-•.;■ ': ■ \. . -■^: 




uc i 'yto"v .^vm' • ■•- ' : ■ 







(0,1) - all blmry wnrds of Iragtb 



{0,1,X)^ - All blnil^Fiilte%#'^i«6#^4. 

((((0.1)^)^)V- (0,1)^ -all binary norda of 

length 16. 



flt 






fV^ ^- ii.v^V .*> :' ;*;i^s^ ^; ^ 



•■:.i'i'?®t.3i 



Important example: 



(01) = 0101."01 = 



2n 



= [0,1] " - (1.{0,1,\} " u 
{0,1,\)^''«0 u 



{0,1, \} .{00,11}.{0,1,\} ") 



all binary words of length 2n which do not 

(1) start wrong 

or (2) end wrong 

or (3) move wrong (contain a forbidden 
local pattern) 



Problem : Given two ex pressions involving letters 
in S,^, and operations 

concatenation 



" U 
" 2 

" n 



union 

squaring 

intersection 



set difference 
is there a way to tell if they describe the same 
language? 
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YES I 
BUT MO GOOD WAY! ! 



Lenaaa . An expression containing n operation 
syndmls describes a subset o£ r 

,n 



(S U \)^ 



Proof . By induction on n: 

If n-0, the expression must consist of a single 
letter or \. 

If E is an expression containing nfl operations, 
then E is of the form 

E.n Ej..^;.(c ,;;':v,/.^ :--.-,';a;;T'Ji;". 

i^iere E., B. Are expressions coirtuiiMli; 'if n operation 
sya^bols. Proof follows lpM41|it||l3r. 



12 For any expression E, let 



13 
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£(E) c: E be the language described by E. 
Remark : Formally, E = E means that E and E„ 
are identical expressions. E and E are 
equivalent (written E = E ) iff 

£(E^) = XCE^). 



E^ 3 E^ iff (E^ - E^) U (E2 - E^) = 



Hence sufficient to test whether an expression 
describes the empty set. 



To test if JJCE) = 0, convert E to a ILst of the 

words in Jb(E) beginning at the "innermost" 
subexpressions of E and working out. 

See if the list is empty when you finish. 

Difficulty: The list for 



2 2 2 

(...(((0 u 1)^) }' - 



"U;:^ 



contains 



15 2" . 2 bits 



16 ': ' : - 

Theorem 1 . There is (for a^ finite S) a 
cons tant ^ > ^ iB4. * "^iv^ nauchiiM SR B\xcAt 
that 

(1) SR acCMts an input w iff w is a well- 
. ^ fonM^^i^ii^u|ion and X^j^ ;^ ^i-..- 

(2) yn) -^- »ax{1^(x) | i(x>!*f Ji^iS 



2 



Thcorta 2 . There is a finite S and a constant 

if SR it any T.M. acceptiiig precisely the 
expressions over S describing the empty 



for ': 'i»|AMllMily mm^-^-n. 



i-..,..r 



(That is. (E I X(E) - 0} ^ Tim(i' ' >1 



,.'-! 



18 To prove Theorem 2: 

(i) Define a relation on languages 

with intuitive meaning that L is easy 
to decide given L„- 
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n^ 



(ii) Show that for any L € Time (2 ) 



L < [E \ Z(E) = 0). 

(iii) Deduce from the Compression Theorem that 
there is an L £ Time (2 ) which is hard 
to decide. 

(iv) Conclude that [E [ ^(E) = 0] is hard to 
decide. 



Def. For L cr L , L c £„ we say L^ < L^ 

(L is polynomial time reducible to L ) iff 



there exists a function f : E, -> E„ 

(1) f is computable in time bounded by a 
polynomial in the length of its argument 

(f € Time( po i, ) where p:N -> N is a 

polynomial and jJ:E -> N is the length 
function), 

(2) X € L^ « f(x) € L^ for all x 6 Ly 



20 Lemma. Let t:K-» N be nondecreaslng, and t(n) 

^ n. 

If L^ < L and L^ € Time(t(n)), then L^ € 

Time(t(p(n))) for some polynomial p:N -♦ H 

n/4 
Contraposltlvc . If L. ^ TimeC 2 ) and L. 

< L , then 3 k > i itosli that Lj f Tiae( 2^ ) 



21 Kx'SjU 

Because 2 1« time-hoiMMit^ the conpresslon 

theorem lnq>llM 3 L. such that 

n/4 
L,€ Ttme(2") - Time(2 ). 

Thm. 2 follows Inaediately from the preceding 
contrapositlve If we show 



22 



Main Construction for Theorem 2.- 

Lemaa . For any h € Time(2'*), there is an 

alphabet S such that 

L < (E I E is an expression over E and 
fi(E) - 0) . 



?'^' ■'^>'%'T^ 



|,fr^^»'V • ""* ^-■^ ^ ^tS'^F-j^jn^l^Jigtsiq 
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..Wii.0llii1tni, iM.fitrwi iti'iiiii .:' 

-■'-'■■'■'-' -■-^■~ ■ ■ - . .- ■ ■■■•' ■- - - ■■■- 

c 2 stc^, cat 

■f»>-;iiHjlii Jt(iillt:.^..|».j3&«.;...: 



!i. iOj •*: 






24 Let Q be the states (boxes In the flowchart) of 

let W be the tape symbols of 3K Including b € W 
f or t^e lilaidc tape symbol, let # be still 
atiother symbol. 

E-*^* Q UWU(#}. 
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For X € A , je(x)"n, 

* 
Conp(x) 6 7j is io be: 



2° /^ \ ,2° 



# b . f start J • X b #(tape after one step)#«** 

• •• # (tape after k steps) # (tape after kfl steps) #*•' 



4 tap» ^hri<^ <yB>bols # 



,n 



Exactly 2*2 + iH-1 symbols between successive #'s. 
i(Coiiip(x)) i 2^<'^^^ 2*- N 
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Comp(x) has the property that any four consecutive 
letters determine the letter 2»2 + n to their right; 




2-2^ + nfl 

Let F = [(a^,a^,aya^,a^)\ ct^ is not the letter 

determined by a,cr CT^cr,}. This follows from the fact that at 
any step the next move of ^ is determined by the state and 
the tape symbol being scanned. 
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Coinp(x) = (starts right)n 
(ends right)n 
( (S U ^) - (moves wrong)) 



,11 



^n 



2" /-- 2 N 

starts right: #b • (start) "X'b • # • (S U >^) 

ends right: (S U X)^' {halt^ . (L U ?^)^ • # 
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moves wrong : 

(L U X)"" . ( U(a^a2a3a^ S^*^ +''"^.a^)). (S U M 



N 



29 Let Rejects (x) = 
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(S U m" • Aalt") . (L -(!])• (S U M^« 



Then 

X 6 L « iUl halts reading a 1 

» Comp(x) n Rejects(x) = 0. 
But expressions for Comp(x) and Rejects(x) 
can be constructed in polynomial time in 
je(x), so L < (E over S | £(E) = 0}. 

Q.E.D. 



Remarks : (1) Thm. 2 holds for expressions 

using only " • ", " U ", " 2" and letters 0,1 . 

(2) If we allow {0,1} to be used in 
expressions Stockmeyer has shown that 



[E with (0,1) I £(E) = 0} € Time ^' 



n 



2n 



but ^ Time ^' h G'loggn 

for some fixed 6 > 0. 

(3) If we allow only " U ", " • " , the 
equivalence problem is complete in*)lp 
(discussed in Karp's lecture). 



31 Remark : Most decidable theories studied in 

mathematical logic require exponential time or 
worse. (An important exception being the 
prepositional calculus, for which lower bounds 
larger than a polynomial are unknown.) 
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Open problems : 

(1) Can the satisfiable formulas of the prepo- 
sitional calculus be recognized in 
polynomial time? (This is the p =71^ 
question of Cook and Karp) . 

(2) Can a multi-tape Turing machine multiply 
integers (in binary notation) in linear 
time? 



33 (3) What is the relation between time and space: 

Known: S^(n) ^ T^(n) ^ ^S^(n) 

(c > 1 depends on iDI) 

Open: If L € Time(2") is L 6 Space(n)? 

(4) Is Space(n) = Nondeterministic Space(n)? 
(The LBA problem of Myhill) 



RjBgrB«s:-?i«^aB(i*!HI' "^ ~7q- ^^i?^7 



^■•- 1 
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(5) Are linear time 3 tape T.M. 's more 
powerful than linear time 2- tape f.M. 's? 

(6) Can the primes (represeiit«i in biniXf) 
be recognized In linear time? 

enti the eotiieixt- frei langwMRgea ? 

(7) C«tt eJN) lixn be miiltlplled iti 
proportional to n ^rlthmetlb 

operations t 

2»ft 
(n is known to be pos#ibl».) 
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